From looking at the steam forums, 3 seems like a really bad port with terrible controller support. DMC4 sounds like a great port, with extra features and modes. I'm thinking of just grabbing 4, though I've never really played games of this type. (maybe 1 hour on DMC back on PS2 long time ago)
You absolutely should get four. Devil May Cry is very similar to Bayonetta in its combat mechanics (completely different to Ninja Gaiden, and Metal Gear Rising in its style) if you've ever played it, and the fourth game has an absolutely fantastic combat system. The near perfect combat system for the style is brought down by what I find rather poor enemy and level design, and a frustrating amount of 'padding' (this is kind of hard to explain without giving away what the second half of the game is), but it makes it an incredibly deep, fun game to play. It runs surprisingly well (my rather meagre laptop maintains a stable 60FPS+ which I was absolutely shocked by) and is quite clean from a visual perspective (it won't blow you away or be a showcase, but it's not ghastly to look at).
There will be a learning curve given that you've little experience with action games of its ilk, seemingly, but this is certainly going to be a good game to introduce yourself to the genre. Personally, I prefer Ninja Gaiden's style of combat to the 'juggling' of Bayonetta and Devil May Cry, but I know there are many who don't share this view (and despite being a preference, if you enjoy the genre I'm sure most are not going to be completely averse to another style of combat) and, as a result, this should not put you off the game in any fashion. If you put in some amount of time learning the mechanics, you'll have introduced yourself to a genre which carries with it a significant amount of freedom in playstyle and replayability. It certainly isn't for everybody, that much is true, but at the price it's at it's very difficult to consider it a catastrophic loss should you not actually like it (and even if this is the case, please don't let it put you off the genre as Rising and Ninja Gaiden in particular are quite distinct from each other, and from Devil May Cry's combat).