It's not that difficult IMO. But there were too things that aggravated me:
1) Cavalry unit control is awkward, to the point that I used them as little as possible. You can't simply switch to a cavalry unit, select the enemy unit you want them to attack and be done with it, like any other unit. If you do this, the cavalry unit will engage the enemy in melee, which is something you would almost never want a cavalry unit to do. Instead, you need to set waypoints around the unit you want the cavalry unit to attack and hope they line up properly to make charging passes through them. Most of the time, the cavalry units would end up turning wide and missing completely. And all of this time spent giving orders to my cavalry was extra time I wasn't controlling my main unit, which brings me to my second point...
2) Your units can't melee for shit unless you're directly controlling them. Your main unit will plow through enemy after enemy, but as soon as you switch over to start issuing commands to your other units, it's as if they just stand there while the enemy beats them to a pulp. Generally I would use my main unit for melee and use archers, sappers, flying units and siege weapons for all my other slots.
I hope they fix those two things up with the sequel.