Integration by parts and representation of information functionals

We introduce a new formalism for computing expectations of functionals of arbitrary random vectors, by using generalised integration by parts formulae. In doing so we extend recent representation formulae for the score function introduced in Nourdin, Peccati and Swan (JFA, to appear) and also provide a new proof of a central identity first discovered in Guo, Shamai, and Verd{\'u} (IEEE Trans. Information Theory, 2005). We derive a representation for the standardized Fisher information of sums of i.i.d. random vectors which use our identities to provide rates of convergence in information theoretic central limit theorems (both in Fisher information distance and in relative entropy).