Manifold boundaries give "gray-box" approximations of complex models

We discuss a method of parameter reduction in complex models known as the Manifold Boundary Approximation Method (MBAM). This approach, based on a geometric interpretation of statistics, maps the model reduction problem to a geometric approximation problem. It operates iteratively, removing one parameter at a time, by approximating a high-dimension, but thin manifold by its boundary. Although the method makes no explicit assumption about the functional form of the model, it does require that the model manifold exhibit a hierarchy of boundaries, i.e., faces, edges, corners, hyper-corners, etc. We empirically show that a variety of model classes have this curious feature, making them amenable to MBAM. These model classes include models composed of elementary functions (e.g., rational functions, exponentials, and partition functions), a variety of dynamical system (e.g., chemical and biochemical kinetics, Linear Time Invariant (LTI) systems, and compartment models), network models (e.g., Bayesian networks, Markov chains, artificial neural networks, and Markov random fields), log-linear probability distributions, and models with symmetries. We discuss how MBAM recovers many common approximation methods for each model class and discuss potential pitfalls and limitations.