Yeah but what are you going to do when your skills are off cooldown?
Also skills that are on cooldown are more designed to fight elite packs than clearing out white mobs quickly. Stuff like Seven Sided Strike and Inner Sanctuary of a Monk you don't waste that on some random white mobs you use it on a big pack or a champ/elite. Most of your time in this game is spend clearing trash mobs... you can't rely on cooldown skills for this job.
It's fine having 2 maybe even 3 cooldown skills but having your entire strategy rely on cooldown skills is not a good idea. Generally speaking the game is designed this way for every character:
You have one resource generator (not a cooldown skill)
You have one skill that you use to dump your resource on (not a cooldown skill)
You have one skill that is for defensive/mobility purposes (cooldown skill)
You have one skill that is on cooldown which is pretty powerful
You have on skill that is like an aura/companion that you have on for a long duration (passive but most have an active component with a short cooldown)
You have one skill that is a good control controller, keeps enemies in position. (can be on a cooldown, some aren't like Cyclone Strike)
Crusader is actually good at 4 of those, maybe even 5 (his Wrath moves aren't that bad). The problem is really that his resource generation is bad which in turn makes his Wrath moves bad as well as he can't spam them as much. This is also compounded by the fact that he can't dual wield so he can't attack fast and even if he could dual wield he would get destroyed due to no 30% damage mitigation.