I'm having a blast with this game and have no problem whatsoever with the controls. I'm at Rhodes right now, but advancing very slowly since I enjoy hunting a lot and the different things I find throughout the world (Had a fun time watching the show at Saint Denis's Cinema). One thing I disliked was the long travel to get to the Trapper, but I found another one at Saint Denis so problem solved. I certainly like how there are no shortcuts since many of the interesting things that have happened to me in game that are not related to the story happened in those moments.
I don't know if you guys are playing without the aim assistance (the option that is default in consoles for aiming), but I have no issues with it be it on the ground or horseback. I consistently headshot enemies in horse persecutions and have only used Dead Eye for hunting. You just have to tilt the stick towards where the enemies head and most of the time you get the headshot. It is way too easy in fact, so if you guys are having issues with the autoaim assist all I can say is git gud fellas.
The controls were muddy for the initial snow area, but after that they are joy to control IMO. This is especially true after coming off The Witcher 3, now that game controls like shiiiiiiit.