Maximum entropy low-rank matrix recovery

We propose in this paper a novel, information-theoretic method, called MaxEnt, for efficient data acquisition for low-rank matrix recovery. This proposed method has important applications to a wide range of problems, including image processing and text document indexing. Fundamental to our design approach is the so-called maximum entropy principle, which states that the measurement masks which maximize the entropy of observations, also maximize the information gain on the unknown matrix $\mathbf{X}$. Coupled with a low-rank stochastic model for $\mathbf{X}$, such a principle (i) reveals novel connections between information-theoretic sampling and subspace packings, and (ii) yields efficient mask construction algorithms for matrix recovery, which significantly outperforms random measurements. We illustrate the effectiveness of MaxEnt in simulation experiments, and demonstrate its usefulness in two real-world applications on image recovery and text document indexing.