The damage is totally fine as far as I went with it (about half way through the game). Some bosses are tricky due to the casting time but if you learn their patterns and when to attack it's doable. I didn't find it particularly enjoyable though but I'm not sure if pure caster is really my kind of thing.
I think the problem was more the late game bosses, but I could be wrong. Not seeking to dissuade anybody from playing it, it's just that it feels like Hard Mode.
I really like the mana system like Demon's Souls had instead of the limited spell uses and having to stack multiple copies of spells in different slots to get more. And the new flask system is cool because it gives you another strategic choice to make. Its just risk versus reward whether you want to cast or heal more. I hated the spell use system honestly. Can't you increase spell casting speed with some stat?
About the more aggressive and fast enemies, it just seems FROM is in a damned if they do or damned if they don't situation. If they don't make changes like that a bunch of people just complain that the game is easy mode compared to other games, but it will make certain builds even more difficult overall.
I don't know, to me spellcasters should be a class where you start off super weak and by the end you are crazy powerful. Hopefully scaling is really good. If it was easy mode all the way through and you never had to use your weapon at all that doesn't seem right to me, and would seem to be a nightmare to balance.
Eh, I feel that spellcasting should be balanced from start to end - I don't want to feel OP, just pew-pewing enemies to death, but neither do I want to have magic just be my conversation starter.
The risk-reward thing is a good point, and From are essentially daring casters to use casting range to their advantage. But I think the problem is that From upped the speed and aggressiveness without giving magic users the speed bonuses or range increase over previous games to deal with that.
An early spell, for example, is a faster, spammable Soul Arrow that does ridiculously low damage. As an aside, I always felt mages should get more tactical weapons instead of just more soul projectiles: clones that distract PvE enemies, a 3-second super armor, that sort of thing.
And re: stats, Sanctuary shared the video: a ring you get boosts cast speed nearly to the cap. Whether that's enough to make a significant in-game difference, I don't know.