SF4 was hard on casuals because it was so-execution based and didn't have a lot of damage, so it was a perfect storm of them needing lots of execution to be good that it made unapproachable for a lot of people.
I'm late to the discussion, but as someone who pegs themselves as a "casual" player I agree with this. My biggest problem is that I'll dedicate as much time (or in cases like a Souls game, more) to other games and I have a habit of only playing one at a time that dates back to the days of SNES. I found that with SF4 in particular that I would take long stretches away and then, when I jump back in, spending an exorbitant amount of time doing execution reps in training was a requirement. And that isn't to say that it's truly any more difficult than past SF games but as I've gotten older... well, the struggle is most certainly real. I've hit my thirties now and muscle memory, twitch reflexes, and even game knowledge (although, watching tourneys helps to minimize this deterioration) goes bye bye much quicker than it used to. Also, having less free time means that I want to maximize my fun which figuring out stuff in training mode can definitely be, but when you need it to drill back to what I consider competent execution standards it kind of sucks.
I'm attracted to fighting games for that unmistakeable rush of competition and that distinct feeling of constant learning. No other games have successfully replicated this for me. (and I've made a good chunk of money competing in and running FPS tourneys) After believing you have a matchup figured out, someone comes along who plays a character unlike anything you've seen. It's fucking awesome and motivating beyond anything else I've experienced in gaming. Nothing is ever truly "figured out" and, therefore, the game is never truly conquered/completed/finished/beaten/etc. There's always something worthwhile to go back to. I just don't enjoy the part of it that feels like a job/chore and feeling like I need to break through a wall to find what I find satisfying doesn't make the destination feel anymore rewarding because it's a process I've already been through.
Digressing back to lack of upkeep on my execution, I recently started playing Ultra again within the past few weeks after taking a hiatus a few months after it released. My motivation being, of course, the itch that the SFV beta left to scratch. SFV was honestly a breath of fresh air in that I was able to jump in cold and hit links consistently after only a few minutes. (thanks, frame buffer!) Within about ten minutes, I was launched into my first match where I blew up a special-happy Bison with Ryu fundamentals. He was going for nothing but big damage and, if I didn't know how to defend against it or he knew how to set it up, then I would've deserved to lose. It certainly wasn't the most satisfying/thrilling victory, but I confidently knew that I won because I had better game knowledge and could take a bit of satisfaction in the victory because of it. More importantly, however, I felt comfortable playing in spite of it being my first match because I wasn't overtly worried that I would drop anything -- and so I had a fun time instead of a stressful one.
In my latest stint of Ultra, I have yet to actually play a match against anyone. Given I'm not putting long stretches of time into it, but I just haven't hit that comfort zone with neither stick nor pad. Hell, there's even a chance that I'm past the point of no return and now I never will. I'll be forced to scrub it out until I drop the game completely. I don't really care if I'm taking a bunch of Ls, but it's when you know how to do something but feel your body working against you that things get frustrating/discouraging. And yes, I certainly get that some like strict execution and derive satisfaction from doing what many others physically can't, but as I've stated above I love fighting games for a different reason and I know that I'm not alone in that. And damn, in dicking around with Omega more I'm finding myself actually having fun learning characters that I never cared to touch in the 4 series before.
Regarding damage specifically, I don't necessarily need or even want it to be a part of V. If it is, then cool. It makes in-the-moment decision making all the more important. If not, then that just means I'll be afforded more opportunities to adjust as the match goes on. Basically, I perceive quick matches as being several comparatively smaller learning opportunities vs a long match being one big learning opportunity. Either is totally cool with me and both bring a different level of enjoyment, so I'm not too concerned with wherever this lands.
Sorry for the stream of consciousness post. I'm just really looking forward to V and I honestly haven't been this excited for a new fighter since my teens. I feel as though I can actually be a part of the community and stay that way -- on my own terms -- instead of feeling like I'm excessively punished for taking a break to do the other things I enjoy. There absolutely should be a divide between casual play and those who put in more time than anyone else, but I don't think the curve between the two plot points should be anywhere near as steep as it was with 4. That's my own selfish opinion.
...damn I wish my old ass didn't love Street Fighter so much growing up and I liked Rising Thunder's aesthetic more.