Not yet reviewed by Pith; the record is open.

We develop a theoretical framework to characterize the decoherence dynamics due to multi-photon scattering in an all-optical switch based on Rydberg atom induced nonlinearities. By incorporating the knowledge of this decoherence process into optimal photon storage and retrieval strategies, we establish optimised switching protocols for experimentally relevant conditions, and evaluate the corresponding limits in the achievable fidelities. Based on these results we work out a simplified description that reproduces recent experiments [arXiv:1511.09445] and provides a new interpretation in terms of many-body decoherence involving multiple incident photons and multiple gate excitations forming the switch. Aside from offering insights into the operational capacity of realistic photon switching capabilities, our work provides a complete description of spin wave decoherence in a Rydberg quantum optics setting, and has immediate relevance to a number of further applications employing photon storage in Rydberg media.