Special points on the product of two modular curves

We prove, assuming the Generalized Riemann Hypothesis for imaginary quadratic fields, that irreducible curves in the product of two modular curves that contain infinitely many complex multiplication points are either a Hecke correspondence or a fibre for one of the two projections. This gives evidence for a conjecture of Oort that says that irreducible components of the Zariski closure of a set of CM points in a Shimura variety are sub Shimura varieties.