Don't think the arrows have any cooldown, but I haven't really tried them. In typical RPG hoarder fashion, I have like 5 thousand special arrows and have never shot one, I keep them for when I might "need" them. Mind you, that as a rogue, so I don't shoot very often to begin with. Still yeah I'd use the charm arrows if I ran into a fight I couldn't beat otherwise to get more charms going. So far hadn't had an issue but some fights are kinda rough.
As for skill points and builds and optimizing, you can definitely optimize but I'd say other than the mages, you'll end up with plenty of spare points, you might not get everything but the effect is going to be minor, for example getting from 4 to 5 in a weapon costs 5skill points and only increases dmg by 10% so it's no big deal if you have a bunch of 1pt stuff instead. I'm still optimizing cause it's in my nature but for example my rogue who's tearing stuff apart currently is floating 17skill points(waiting for the special weapon skill thing to unlock to dump 15points in that) as well as 4 attributes(not sure where I want them so keeping them until I figured something good), so you have a lot of room. Proper combat strategy, positionning and tactics is a lot more important than builds or gear(to an extent, for rogues you do want those +1 sneak items but that's it really).
Mages have less spare points though because you do want to level the skills high to get more slots and stuff. If you have 2mages it's easier but if you're going for a single mage then you'll need every point possible to try to max 1school of magic while having all the good cheap stuff from the others(or if you play lone wolf, max 2).