I've really enjoyed using this carriage mod which adds carriage services to all of the major cities in Skyrim: http://skyrim.nexusmods.com/mods/8544
I started using this when I realized that fast traveling was ruining the game for me. Now, I try to avoid fast traveling unless I'm completing a quest. So far, it's made the game so much more enjoyable.
I am not using that MOD, but I have been doing the same thing since installing Frostfall. If I am in the middle of nowhere without a horse, I'm walking. It is definitely making the game take a bit longer, but I have seen stuff I have skipped over before in my previous play through.
I think I am going to have to install that carriage mod though. It makes sense that the cities would all have a carriage service in some form.
Someone needs to write a rent a horse mod that allows you to rent a horse from any stable.