I wouldn't be stepping aside primarily for the sake of the horse: if I moved aside, it would be because there was a very large animal that I couldn't keep an eye on coming up behind me. I don't want to risk being stepped on or jostled if I stop for some reason, or slow down when the rider isn't expecting me to.

"Don't make the horse think you're lurking" is a request I will try to remember. However, paths aren't always well laid out for that. I won't lurk behind a tree on purpose, but if the path isn't wide enough for me to stand both on the path and safely out of the horse's way, there might not be anywhere reasonable to stand that wouldn't put a tree between me and the horse at some point.