Smooth surface systems may contain smooth curves which have no measure of maximal entropy

In this paper, we study Borel probability measures of maximal entropy for analytic subsets in a dynamical system. It is well known that higher smoothness of the map over smooth space plays important role in the study of invariant measures of maximal entropy. A famous theorem of Newhouse states that smooth diffeomorphisms on compact manifolds without boundary have invariant measures of maximal entropy. However, we show that the situation becomes completely different when we study measures of maximal entropy for analytic subsets. Namely, we construct a smooth surface system which contains a smooth curve having no Borel probability measure of maximal entropy. Another evidence to show this difference is that, once an analytic set has one measure of maximal entropy, then the set has many measures of maximal entropy (no matter if we consider packing or Bowen entropy). For a general dynamical system with positive entropy $h_\mathrm{top}(T)$, we shall show that the system contains not only a Borel subset which has Borel probability measures of maximal entropy and has entropy sufficiently close to $h_\mathrm{top}(T)$, but also a Borel subset which has no Borel probability measures of maximal entropy and has entropy equal to the arbitrarily given positive real number which is at most $h_\mathrm{top}(T)$. We also provide in all $h$-expansive systems a full characterization for analytic subsets which have Borel probability measures of maximal entropy. Consequently, if let $Z\subset \mathbb{R}^n$ be any analytic subset with positive Hausdorff dimension in Euclidean space, then the set $Z$ either has a measure of full lower Hausdorff dimension, or it can be partitioned into a union of countably many analytic sets $\{Z_i\}_{i\in \mathbb{N}}$ with $\dim_{\mathcal{H}}(Z_i) < \dim_{\mathcal{H}}(Z)$ for each $i$.