Why do we remember the past and not the future? The 'time oriented coarse graining' hypothesis

Phenomenological arrows of time can be traced to a past low-entropy state. Does this imply the universe was in an improbable state in the past? I suggest a different possibility: past low-entropy depends on the coarse-graining implicit in our definition of entropy. This, in turn depends on our physical coupling to the rest of the world. I conjecture that $any$ generic motion of a sufficiently rich system satisfies the second law of thermodynamics, in either direction of time, for $some$ choice of macroscopic observables. The low entropy of the past could then be due to the way we couple to the universe (a way needed for us doing what we do), hence to our natural macroscopic variables, rather than to a strange past microstate of the world at large.