Transforming gravity: from derivative couplings to matter to second-order scalar-tensor theories beyond the Horndeski Lagrangian

We study the structure of scalar-tensor theories of gravity based on derivative couplings between the scalar and the matter degrees of freedom introduced through an effective metric. Such interactions are classified by their tensor structure into conformal (scalar), disformal (vector) and extended disformal (traceless tensor), as well as by the derivative order of the scalar field. Relations limited to first derivatives of the field ensure second order equations of motion in the Einstein frame and hence the absence of Ostrogradski ghost degrees of freedom. The existence of a mapping to the Jordan frame is not trivial in the general case, and can be addressed using the Jacobian of the frame transformation through its eigenvalues and eigentensors. These objects also appear in the study of different aspects of such theories, including the metric and field redefinition transformation of the path integral in the quantum mechanical description. Although sane in the Einstein frame, generic disformally coupled theories are described by higher order equations of motion in the Jordan frame. This apparent contradiction is solved by the use of a hidden constraint: the contraction of the metric equations with a Jacobian eigentensor provides a constraint relation for the higher field derivatives, which allows one to express the dynamical equations in a second order form. This signals a loophole in Horndeski's theorem and allows one to enlarge the set of scalar-tensor theories which are Ostrogradski-stable. The transformed Gauss-Bonnet terms are also discussed for the simplest conformal and disformal relations.