Can all cosmological observations be accurately interpreted with a unique geometry?

The recent analysis of the Planck results reveals a tension between the best fits for ({\Omega}m0, H0) derived from the cosmic microwave background or baryonic acoustic oscillations on the one hand, and the Hubble diagram on the other hand. These observations probe the universe on very different scales since they involve light beams of very different angular sizes; hence the tension between them may indicate that they should not be interpreted the same way. More precisely, this Letter questions the accuracy of using only the (perturbed) Friedmann-Lema\^itre geometry to interpret all the cosmological observations, regardless of their angular or spatial resolution. We show that using an inhomogeneous "Swiss-cheese" model to interpret the Hubble diagram allows to reconcile it with the Planck results. Such an approach does not require us to invoke new physics nor to violate the Copernican principle.